{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "481f19d5",
   "metadata": {},
   "source": [
    "# Python机器学习Kaggle案例实战（第21期）第7课书面作业\n",
    "学号：113778\n",
    "\n",
    "**作业内容**  \n",
    "1. 阅读《Ad Click Prediction: a View from the Trenches》，简述论文的基本方法  \n",
    "2. 安装并使用简单的数据集测试LibFM或LibFFM"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "246d856f",
   "metadata": {},
   "source": [
    "## 1.作业1"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b7ca4315",
   "metadata": {},
   "source": [
    "论文讲解的是点击率预估的核心算法，因为谷歌要处理的数据集通常都非常庞大，不管是样本数量还是样本的特征数都是百亿级别的，所以选用什么样的算法至关重要，在这种情况下谷歌工程师选择了逻辑回归，这是一个非常传统但也非常强大的线性分类工具，而且效率还比较高。  \n",
    "逻辑回归是要对二元分类问题进行建模，模型的核心是通过一组特征以及所对应的参数来对目标进行拟合。这个拟合的过程是通过一个叫逻辑函数来完成的，使得线性的特征以及参数的拟合能够非线性转换为二元标签。  \n",
    "普通的逻辑回归并不适应大规模的广告点击率预估。有两个原因：\n",
    "1. 数据量太大。传统的逻辑回归参数训练过程都依靠牛顿法（Newton’s Method）或者 L-BFGS 等算法。这些算法并不太容易在大规模数据上得以处理。\n",
    "2. 不太容易得到比较稀疏（Sparse）的答案（Solution）。也就是说，虽然数据中特征的总数很多，但是对于单个数据点来说，有效特征是有限而且稀疏的。\n",
    "\n",
    "我们希望最终学习到的模型也是稀疏的，也就是对于单个数据点来说，仅有少量特征是有数据的。传统的解法，甚至包括一些传统的在线逻辑回归，都不能很好地处理稀疏性问题。  \n",
    "这篇文章提出了用一种叫FTRL（Follow The Regularized Leader）的在线逻辑回归算法来解决上述问题。FTRL 是一种在线算法，因此算法的核心就是模型的参数会在每一个数据点进行更新。FTRL 把传统的逻辑回归的目标函数进行了改写。  \n",
    "新的目标函数分为三个部分：\n",
    "1. 第一部分是一个用过去所有的梯度值（Gradients）来重新加权（Re-Weight）所有的参数值；\n",
    "2. 第二部分是当前最新的参数值尽可能不偏差之前所有的参数值；\n",
    "3. 第三个部分则是希望当前的参数值能够有稀疏的解（通过 L1 来直接约束）。\n",
    "从这三个部分的目标函数来看，这个算法既能让参数的变化符合数据规律（从梯度来控制），也能让参数不至于偏离过去已有的数值，从而整个参数不会随着一些异常的数据点而发生剧烈变化。  \n",
    "在算法上另外一个比较新颖的地方，就是对每一个特征维度的学习速率都有一个动态的自动调整。传统的随机梯度下降（Stochastic Gradient Descent）算法或是简单的在线逻辑回归都没有这样的能力，造成了传统的算法需要花很长时间来手工调学习速率等参数。  \n",
    "同时，因为每一个特征维度上特征数值的差异，造成了没法对所有特征选取统一的学习速率。而 FTRL 带来的则是对每一个维度特征的动态学习速率，一举解决了手动调整学习算法的学习速率问题。简单说来，学习速率就是根据每一个维度目前所有梯度的平方和的倒数进行调整，这个平方和越大，则学习速率越慢。\n",
    "\n",
    "另外，光有一个比较优化的在线逻辑回归算法，依然很难得到最好的效果，还会有很多细小的系统调优过程。  \n",
    "比如文章介绍了利用布隆过滤器（Bloom Filter）的方法，来动态决定某一个特征是否需要加入到模型中。虽然这样的方法是概率性的，意思是说，某一个特征即便可能小于某一个值，也有可能被错误加入，但是发生这样事件的概率是比较小的。通过布隆过滤器调优之后，模型的 AUC 仅仅降低了 0.008%，但是内存的消耗却减少了 60% 之多，可见很多特征仅仅存在于少量的数据中。  \n",
    "文章还介绍了一系列的方法来减少内存的消耗。比如利用更加紧凑的存储格式，而不是简单的 32 位或者 64 位的浮点数存储。作者们利用了一种叫 q2.13 的格式，更加紧凑地存储节省了另外 75% 的内存空间。这一点上还是让人印象深刻的。  \n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "db78801c",
   "metadata": {},
   "source": [
    "## 2. 作业2\n",
    "安装并使用简单的数据集测试LibFM或LibFFM。\n",
    "\n",
    "我还是尝试安装了xlearn库，同时支持FM和FFM模型。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "fff7f8d0",
   "metadata": {},
   "outputs": [],
   "source": [
    "import xlearn as xl"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "4086efee",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'0.4.4'"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "xl.__version__"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "edd74c5a",
   "metadata": {},
   "source": [
    "### 2.1 FM模型\n",
    "采用Kaggle中的Titanic数据集，预测生存情况，二分类问题。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "8ccf3add",
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "import pd2ffm as pf\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.metrics import accuracy_score"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "f34506da",
   "metadata": {},
   "outputs": [],
   "source": [
    "train=pd.read_csv('train.csv')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "6cf34568",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PassengerId</th>\n",
       "      <th>Survived</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Name</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Ticket</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Cabin</th>\n",
       "      <th>Embarked</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Braund, Mr. Owen Harris</td>\n",
       "      <td>male</td>\n",
       "      <td>22.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>A/5 21171</td>\n",
       "      <td>7.2500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Cumings, Mrs. John Bradley (Florence Briggs Th...</td>\n",
       "      <td>female</td>\n",
       "      <td>38.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>PC 17599</td>\n",
       "      <td>71.2833</td>\n",
       "      <td>C85</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>Heikkinen, Miss. Laina</td>\n",
       "      <td>female</td>\n",
       "      <td>26.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>STON/O2. 3101282</td>\n",
       "      <td>7.9250</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Futrelle, Mrs. Jacques Heath (Lily May Peel)</td>\n",
       "      <td>female</td>\n",
       "      <td>35.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>113803</td>\n",
       "      <td>53.1000</td>\n",
       "      <td>C123</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Allen, Mr. William Henry</td>\n",
       "      <td>male</td>\n",
       "      <td>35.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>373450</td>\n",
       "      <td>8.0500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>886</th>\n",
       "      <td>887</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>Montvila, Rev. Juozas</td>\n",
       "      <td>male</td>\n",
       "      <td>27.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>211536</td>\n",
       "      <td>13.0000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>887</th>\n",
       "      <td>888</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Graham, Miss. Margaret Edith</td>\n",
       "      <td>female</td>\n",
       "      <td>19.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>112053</td>\n",
       "      <td>30.0000</td>\n",
       "      <td>B42</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>888</th>\n",
       "      <td>889</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Johnston, Miss. Catherine Helen \"Carrie\"</td>\n",
       "      <td>female</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>W./C. 6607</td>\n",
       "      <td>23.4500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>889</th>\n",
       "      <td>890</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Behr, Mr. Karl Howell</td>\n",
       "      <td>male</td>\n",
       "      <td>26.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>111369</td>\n",
       "      <td>30.0000</td>\n",
       "      <td>C148</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>890</th>\n",
       "      <td>891</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Dooley, Mr. Patrick</td>\n",
       "      <td>male</td>\n",
       "      <td>32.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>370376</td>\n",
       "      <td>7.7500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Q</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>891 rows × 12 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "     PassengerId  Survived  Pclass  \\\n",
       "0              1         0       3   \n",
       "1              2         1       1   \n",
       "2              3         1       3   \n",
       "3              4         1       1   \n",
       "4              5         0       3   \n",
       "..           ...       ...     ...   \n",
       "886          887         0       2   \n",
       "887          888         1       1   \n",
       "888          889         0       3   \n",
       "889          890         1       1   \n",
       "890          891         0       3   \n",
       "\n",
       "                                                  Name     Sex   Age  SibSp  \\\n",
       "0                              Braund, Mr. Owen Harris    male  22.0      1   \n",
       "1    Cumings, Mrs. John Bradley (Florence Briggs Th...  female  38.0      1   \n",
       "2                               Heikkinen, Miss. Laina  female  26.0      0   \n",
       "3         Futrelle, Mrs. Jacques Heath (Lily May Peel)  female  35.0      1   \n",
       "4                             Allen, Mr. William Henry    male  35.0      0   \n",
       "..                                                 ...     ...   ...    ...   \n",
       "886                              Montvila, Rev. Juozas    male  27.0      0   \n",
       "887                       Graham, Miss. Margaret Edith  female  19.0      0   \n",
       "888           Johnston, Miss. Catherine Helen \"Carrie\"  female   NaN      1   \n",
       "889                              Behr, Mr. Karl Howell    male  26.0      0   \n",
       "890                                Dooley, Mr. Patrick    male  32.0      0   \n",
       "\n",
       "     Parch            Ticket     Fare Cabin Embarked  \n",
       "0        0         A/5 21171   7.2500   NaN        S  \n",
       "1        0          PC 17599  71.2833   C85        C  \n",
       "2        0  STON/O2. 3101282   7.9250   NaN        S  \n",
       "3        0            113803  53.1000  C123        S  \n",
       "4        0            373450   8.0500   NaN        S  \n",
       "..     ...               ...      ...   ...      ...  \n",
       "886      0            211536  13.0000   NaN        S  \n",
       "887      0            112053  30.0000   B42        S  \n",
       "888      2        W./C. 6607  23.4500   NaN        S  \n",
       "889      0            111369  30.0000  C148        C  \n",
       "890      0            370376   7.7500   NaN        Q  \n",
       "\n",
       "[891 rows x 12 columns]"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "80964d2f",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "PassengerId      0\n",
       "Survived         0\n",
       "Pclass           0\n",
       "Name             0\n",
       "Sex              0\n",
       "Age            177\n",
       "SibSp            0\n",
       "Parch            0\n",
       "Ticket           0\n",
       "Fare             0\n",
       "Cabin          687\n",
       "Embarked         2\n",
       "dtype: int64"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train.isnull().sum()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "726bb21b",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 891 entries, 0 to 890\n",
      "Data columns (total 12 columns):\n",
      " #   Column       Non-Null Count  Dtype  \n",
      "---  ------       --------------  -----  \n",
      " 0   PassengerId  891 non-null    int64  \n",
      " 1   Survived     891 non-null    int64  \n",
      " 2   Pclass       891 non-null    int64  \n",
      " 3   Name         891 non-null    object \n",
      " 4   Sex          891 non-null    object \n",
      " 5   Age          714 non-null    float64\n",
      " 6   SibSp        891 non-null    int64  \n",
      " 7   Parch        891 non-null    int64  \n",
      " 8   Ticket       891 non-null    object \n",
      " 9   Fare         891 non-null    float64\n",
      " 10  Cabin        204 non-null    object \n",
      " 11  Embarked     889 non-null    object \n",
      "dtypes: float64(2), int64(5), object(5)\n",
      "memory usage: 83.7+ KB\n"
     ]
    }
   ],
   "source": [
    "train.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "4a98b504",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Name</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Ticket</th>\n",
       "      <th>Cabin</th>\n",
       "      <th>Embarked</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>891</td>\n",
       "      <td>891</td>\n",
       "      <td>891</td>\n",
       "      <td>204</td>\n",
       "      <td>889</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>unique</th>\n",
       "      <td>891</td>\n",
       "      <td>2</td>\n",
       "      <td>681</td>\n",
       "      <td>147</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>top</th>\n",
       "      <td>Fortune, Mr. Charles Alexander</td>\n",
       "      <td>male</td>\n",
       "      <td>1601</td>\n",
       "      <td>C23 C25 C27</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>freq</th>\n",
       "      <td>1</td>\n",
       "      <td>577</td>\n",
       "      <td>7</td>\n",
       "      <td>4</td>\n",
       "      <td>644</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                  Name   Sex Ticket        Cabin Embarked\n",
       "count                              891   891    891          204      889\n",
       "unique                             891     2    681          147        3\n",
       "top     Fortune, Mr. Charles Alexander  male   1601  C23 C25 C27        S\n",
       "freq                                 1   577      7            4      644"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train.select_dtypes(include=['O']).describe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "a298a13a",
   "metadata": {},
   "outputs": [],
   "source": [
    "train.Age=train.Age.fillna(train.Age.mean())\n",
    "train['Embarked']=train['Embarked'].fillna('S')\n",
    "train['Cabin'] = train['Cabin'].fillna('NO')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "bf5ec679",
   "metadata": {},
   "outputs": [],
   "source": [
    "train=train.drop(columns=['PassengerId','Name'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "71bb93ef",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Survived</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Ticket</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Cabin</th>\n",
       "      <th>Embarked</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>male</td>\n",
       "      <td>22.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>A/5 21171</td>\n",
       "      <td>7.2500</td>\n",
       "      <td>NO</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>female</td>\n",
       "      <td>38.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>PC 17599</td>\n",
       "      <td>71.2833</td>\n",
       "      <td>C85</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>female</td>\n",
       "      <td>26.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>STON/O2. 3101282</td>\n",
       "      <td>7.9250</td>\n",
       "      <td>NO</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>female</td>\n",
       "      <td>35.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>113803</td>\n",
       "      <td>53.1000</td>\n",
       "      <td>C123</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>male</td>\n",
       "      <td>35.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>373450</td>\n",
       "      <td>8.0500</td>\n",
       "      <td>NO</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>886</th>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>male</td>\n",
       "      <td>27.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>211536</td>\n",
       "      <td>13.0000</td>\n",
       "      <td>NO</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>887</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>female</td>\n",
       "      <td>19.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>112053</td>\n",
       "      <td>30.0000</td>\n",
       "      <td>B42</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>888</th>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>female</td>\n",
       "      <td>29.699118</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>W./C. 6607</td>\n",
       "      <td>23.4500</td>\n",
       "      <td>NO</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>889</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>male</td>\n",
       "      <td>26.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>111369</td>\n",
       "      <td>30.0000</td>\n",
       "      <td>C148</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>890</th>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>male</td>\n",
       "      <td>32.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>370376</td>\n",
       "      <td>7.7500</td>\n",
       "      <td>NO</td>\n",
       "      <td>Q</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>891 rows × 10 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "     Survived  Pclass     Sex        Age  SibSp  Parch            Ticket  \\\n",
       "0           0       3    male  22.000000      1      0         A/5 21171   \n",
       "1           1       1  female  38.000000      1      0          PC 17599   \n",
       "2           1       3  female  26.000000      0      0  STON/O2. 3101282   \n",
       "3           1       1  female  35.000000      1      0            113803   \n",
       "4           0       3    male  35.000000      0      0            373450   \n",
       "..        ...     ...     ...        ...    ...    ...               ...   \n",
       "886         0       2    male  27.000000      0      0            211536   \n",
       "887         1       1  female  19.000000      0      0            112053   \n",
       "888         0       3  female  29.699118      1      2        W./C. 6607   \n",
       "889         1       1    male  26.000000      0      0            111369   \n",
       "890         0       3    male  32.000000      0      0            370376   \n",
       "\n",
       "        Fare Cabin Embarked  \n",
       "0     7.2500    NO        S  \n",
       "1    71.2833   C85        C  \n",
       "2     7.9250    NO        S  \n",
       "3    53.1000  C123        S  \n",
       "4     8.0500    NO        S  \n",
       "..       ...   ...      ...  \n",
       "886  13.0000    NO        S  \n",
       "887  30.0000   B42        S  \n",
       "888  23.4500    NO        S  \n",
       "889  30.0000  C148        C  \n",
       "890   7.7500    NO        Q  \n",
       "\n",
       "[891 rows x 10 columns]"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "86045462",
   "metadata": {},
   "outputs": [],
   "source": [
    "train_data=pd.get_dummies(train) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "7276bb7b",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Survived</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Sex_female</th>\n",
       "      <th>Sex_male</th>\n",
       "      <th>Ticket_110152</th>\n",
       "      <th>Ticket_110413</th>\n",
       "      <th>...</th>\n",
       "      <th>Cabin_F2</th>\n",
       "      <th>Cabin_F33</th>\n",
       "      <th>Cabin_F38</th>\n",
       "      <th>Cabin_F4</th>\n",
       "      <th>Cabin_G6</th>\n",
       "      <th>Cabin_NO</th>\n",
       "      <th>Cabin_T</th>\n",
       "      <th>Embarked_C</th>\n",
       "      <th>Embarked_Q</th>\n",
       "      <th>Embarked_S</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>22.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>7.2500</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>38.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>71.2833</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>26.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7.9250</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>35.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>53.1000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>35.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>8.0500</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>886</th>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>27.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>13.0000</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>887</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>19.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>30.0000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>888</th>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>29.699118</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>23.4500</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>889</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>26.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>30.0000</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>890</th>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>32.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7.7500</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>891 rows × 840 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "     Survived  Pclass        Age  SibSp  Parch     Fare  Sex_female  Sex_male  \\\n",
       "0           0       3  22.000000      1      0   7.2500           0         1   \n",
       "1           1       1  38.000000      1      0  71.2833           1         0   \n",
       "2           1       3  26.000000      0      0   7.9250           1         0   \n",
       "3           1       1  35.000000      1      0  53.1000           1         0   \n",
       "4           0       3  35.000000      0      0   8.0500           0         1   \n",
       "..        ...     ...        ...    ...    ...      ...         ...       ...   \n",
       "886         0       2  27.000000      0      0  13.0000           0         1   \n",
       "887         1       1  19.000000      0      0  30.0000           1         0   \n",
       "888         0       3  29.699118      1      2  23.4500           1         0   \n",
       "889         1       1  26.000000      0      0  30.0000           0         1   \n",
       "890         0       3  32.000000      0      0   7.7500           0         1   \n",
       "\n",
       "     Ticket_110152  Ticket_110413  ...  Cabin_F2  Cabin_F33  Cabin_F38  \\\n",
       "0                0              0  ...         0          0          0   \n",
       "1                0              0  ...         0          0          0   \n",
       "2                0              0  ...         0          0          0   \n",
       "3                0              0  ...         0          0          0   \n",
       "4                0              0  ...         0          0          0   \n",
       "..             ...            ...  ...       ...        ...        ...   \n",
       "886              0              0  ...         0          0          0   \n",
       "887              0              0  ...         0          0          0   \n",
       "888              0              0  ...         0          0          0   \n",
       "889              0              0  ...         0          0          0   \n",
       "890              0              0  ...         0          0          0   \n",
       "\n",
       "     Cabin_F4  Cabin_G6  Cabin_NO  Cabin_T  Embarked_C  Embarked_Q  Embarked_S  \n",
       "0           0         0         1        0           0           0           1  \n",
       "1           0         0         0        0           1           0           0  \n",
       "2           0         0         1        0           0           0           1  \n",
       "3           0         0         0        0           0           0           1  \n",
       "4           0         0         1        0           0           0           1  \n",
       "..        ...       ...       ...      ...         ...         ...         ...  \n",
       "886         0         0         1        0           0           0           1  \n",
       "887         0         0         0        0           0           0           1  \n",
       "888         0         0         1        0           0           0           1  \n",
       "889         0         0         0        0           1           0           0  \n",
       "890         0         0         1        0           0           1           0  \n",
       "\n",
       "[891 rows x 840 columns]"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "237f8bb6",
   "metadata": {},
   "outputs": [],
   "source": [
    "ftrain=train_data[:500]\n",
    "ftest=train_data[500:]\n",
    "\n",
    "ftrain.to_csv('fm_train.txt',sep=' ',index=False,header=False)\n",
    "\n",
    "ftest1=ftest.drop(columns=['Survived'])\n",
    "ftest1.to_csv('fm_test.txt',sep=' ',index=False,header=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "ea5edba1",
   "metadata": {},
   "outputs": [],
   "source": [
    "import xlearn as xl\n",
    "\n",
    "# Training task\n",
    "fm_model = xl.create_fm()  # Use factorization machine\n",
    "fm_model.setTrain(\"fm_train.txt\")  # Training data\n",
    "fm_model.setValidate(\"fm_test.txt\")  # Validation data\n",
    "\n",
    "# param:\n",
    "#  0. Binary classification task\n",
    "#  1. learning rate: 0.2\n",
    "#  2. lambda: 0.002\n",
    "#  3. metric: accuracy\n",
    "param = {'task':'binary', 'lr':0.2, \n",
    "         'lambda':0.002, 'metric':'acc'}\n",
    "\n",
    "# Start to train\n",
    "# The trained model will be stored in model.out\n",
    "fm_model.fit(param, './fm_model.out')\n",
    "\n",
    "# Prediction task\n",
    "fm_model.setTest(\"fm_test.txt\")  # Test data\n",
    "#fm_model.setSigmoid()  # Convert output to 0-1\n",
    "fm_model.setSign()\n",
    "\n",
    "# Start to predict\n",
    "# The output result will be stored in output.txt\n",
    "fm_model.predict(\"./fm_model.out\", \"./fm_output.txt\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "921913ba",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "用FM模型预测精度： 0.618925831202046\n"
     ]
    }
   ],
   "source": [
    "pred_data=pd.read_csv('fm_output.txt',header=None)\n",
    "y_pred=pred_data.iloc[:,0]\n",
    "y_true=ftest['Survived']\n",
    "\n",
    "print('用FM模型预测精度：',accuracy_score(y_true, y_pred)) "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c4a5dca7",
   "metadata": {},
   "source": [
    "### 2.2 FFM模型\n",
    "\n",
    "沿用2.1章节的数据集。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "5f5f83a1",
   "metadata": {},
   "outputs": [],
   "source": [
    "class FFMFormatPandas:\n",
    "    def __init__(self):\n",
    "        self.field_index_ = None\n",
    "        self.feature_index_ = None\n",
    "        self.y = None\n",
    "\n",
    "    def fit(self, df, y=None):\n",
    "        self.y = y\n",
    "        df_ffm = df[df.columns.difference([self.y])]\n",
    "        if self.field_index_ is None:\n",
    "            self.field_index_ = {col: i for i, col in enumerate(df_ffm)}\n",
    "\n",
    "        if self.feature_index_ is not None:\n",
    "            last_idx = max(list(self.feature_index_.values()))\n",
    "\n",
    "        if self.feature_index_ is None:\n",
    "            self.feature_index_ = dict()\n",
    "            last_idx = 0\n",
    "\n",
    "        for col in df.columns:\n",
    "            vals = df[col].unique()\n",
    "            for val in vals:\n",
    "                if pd.isnull(val):\n",
    "                    continue\n",
    "                name = '{}_{}'.format(col, val)\n",
    "                if name not in self.feature_index_:\n",
    "                    self.feature_index_[name] = last_idx\n",
    "                    last_idx += 1\n",
    "            self.feature_index_[col] = last_idx\n",
    "            last_idx += 1\n",
    "        return self\n",
    "\n",
    "    def fit_transform(self, df, y=None):\n",
    "        self.fit(df, y)\n",
    "        return self.transform(df)\n",
    "\n",
    "    def transform_row_(self, row, t):\n",
    "        ffm = []\n",
    "        if self.y != None:\n",
    "            ffm.append(str(row.loc[row.index == self.y][0]))\n",
    "        if self.y is None:\n",
    "            ffm.append(str(0))\n",
    "\n",
    "        for col, val in row.loc[row.index != self.y].to_dict().items():\n",
    "            col_type = t[col]\n",
    "            name = '{}_{}'.format(col, val)\n",
    "            if col_type.kind ==  'O':\n",
    "                ffm.append('{}:{}:1'.format(self.field_index_[col], self.feature_index_[name]))\n",
    "            elif col_type.kind == 'i':\n",
    "                ffm.append('{}:{}:{}'.format(self.field_index_[col], self.feature_index_[col], val))\n",
    "        return ' '.join(ffm)\n",
    "\n",
    "    def transform(self, df):\n",
    "        t = df.dtypes.to_dict()\n",
    "        return pd.Series({idx: self.transform_row_(row, t) for idx, row in df.iterrows()})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "b474ff94",
   "metadata": {},
   "outputs": [],
   "source": [
    "ffm_train = FFMFormatPandas()\n",
    "ffm_train_data = ffm_train.fit_transform(train, y='Survived')\n",
    "\n",
    "ffm_train_data[:500].to_csv('ffm_train.txt',index=False,header=False)\n",
    "ffm_test_data=ffm_train_data[500:]\n",
    "ffm_test_data.to_csv('ffm_test_label.txt',index=False,header=False)\n",
    "ffm_test_data.drop(columns=['Survived']).to_csv('ffm_test.txt',index=False,header=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "7133aed1",
   "metadata": {},
   "outputs": [],
   "source": [
    "import xlearn as xl\n",
    "\n",
    "# Training task\n",
    "ffm_model = xl.create_ffm() # Use field-aware factorization machine\n",
    "ffm_model.setTrain(\"./ffm_train.txt\")  # Training data\n",
    "ffm_model.setValidate(\"./ffm_test.txt\")  # Validation data\n",
    "\n",
    "# param:\n",
    "#  0. binary classification\n",
    "#  1. learning rate: 0.2\n",
    "#  2. regular lambda: 0.002\n",
    "#  3. evaluation metric: accuracy\n",
    "param = {'task':'binary', 'lr':0.2, 'lambda':0.002, 'metric':'acc'}\n",
    "\n",
    "# Start to train\n",
    "# The trained model will be stored in model.out\n",
    "ffm_model.fit(param, './ffm_model.out')\n",
    "\n",
    "# Prediction task\n",
    "ffm_model.setTest(\"./ffm_test.txt\")  # Test data\n",
    "#ffm_model.setSigmoid()  # Convert output to 0-1\n",
    "ffm_model.setSign()\n",
    "\n",
    "# Start to predict\n",
    "# The output result will be stored in output.txt\n",
    "ffm_model.predict(\"./ffm_model.out\", \"./ffm_output.txt\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "a9e5a302",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "用FFM模型预测精度： 0.7851662404092071\n"
     ]
    }
   ],
   "source": [
    "pred_data=pd.read_csv('ffm_output.txt',header=None)\n",
    "y_pred=pred_data.iloc[:,0]\n",
    "\n",
    "df_test=pd.read_csv('ffm_test_label.txt',sep=' ', header=None)\n",
    "y_true=df_test[0]\n",
    "\n",
    "print('用FFM模型预测精度：',accuracy_score(y_true, y_pred)) "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3992725d",
   "metadata": {},
   "source": [
    "从结果看FFM模型确实比FM模型要好一点。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "08cdc1a0",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
